Automating the multimodal analysis of musculoskeletal imaging in the presence of hip implants

In patients treated with hip arthroplasty, the muscular condition and presence of inflammatory reactions are assessed using magnetic resonance imaging (MRI). As MRI lacks contrast for bony structures, computed tomography (CT) is preferred for clinical evaluation of bone tissue and orthopaedic surgical planning. Combining the complementary information of MRI and CT could improve current clinical practice for diagnosis, monitoring and treatment planning. In particular, the different contrast of these modalities could help better quantify the presence of fatty infiltration to characterise muscular condition after hip replacement. In this thesis, I developed automated processing tools for the joint analysis of CT and MR images of patients with hip implants. In order to combine the multimodal information, a novel nonlinear registration algorithm was introduced, which imposes rigidity constraints on bony structures to ensure realistic deformation. I implemented and thoroughly validated a fully automated framework for the multimodal segmentation of healthy and pathological musculoskeletal structures, as well as implants. This framework combines the proposed registration algorithm with tailored image quality enhancement techniques and a multi-atlas-based segmentation approach, providing robustness against the large population anatomical variability and the presence of noise and artefacts in the images. The automation of muscle segmentation enabled the derivation of a measure of fatty infiltration, the Intramuscular Fat Fraction, useful to characterise the presence of muscle atrophy. The proposed imaging biomarker was shown to strongly correlate with the atrophy radiological score currently used in clinical practice. Finally, a preliminary work on multimodal metal artefact reduction, using an unsupervised deep learning strategy, showed promise for improving the postprocessing of CT and MR images heavily corrupted by metal artefact. This work represents a step forward towards the automation of image analysis in hip arthroplasty, supporting and quantitatively informing the decision-making process about patient’s management

Deep Boosted Regression for MR to CT Synthesis

Attenuation correction is an essential requirement of positron emission tomography (PET) image reconstruction to allow for accurate quantification. However, attenuation correction is particularly challenging for PET-MRI as neither PET nor magnetic resonance imaging (MRI) can directly image tissue attenuation properties. MRI-based computed tomography (CT) synthesis has been proposed as an alternative to physics based and segmentation-based approaches that assign a population-based tissue density value in order to generate an attenuation map. We propose a novel deep fully convolutional neural network that generates synthetic CTs in a recursive manner by gradually reducing the residuals of the previous network, increasing the overall accuracy and generalisability, while keeping the number of trainable parameters within reasonable limits. The model is trained on a database of 20 pre-acquired MRI/CT pairs and a four-fold random bootstrapped validation with a 80:20 split is performed. Quantitative results show that the proposed framework outperforms a state-of-the-art atlas-based approach decreasing the Mean Absolute Error (MAE) from 131HU to 68HU for the synthetic CTs and reducing the PET reconstruction error from 14.3% to 7.2%.Comment: Accepted at SASHIMI201

Individualized Prognostic Prediction of the Long-Term Functional Trajectory in Pediatric Acquired Brain Injury

In pediatric acquired brain injury, heterogeneity of functional response to specific rehabilitation treatments is a key confound to medical decisions and outcome prediction. We aimed to identify patient subgroups sharing comparable trajectories, and to implement a method for the early prediction of the long-term recovery course from clinical condition at first discharge. 600 consecutive patients with acquired brain injury (7.4 years ± 5.2; 367 males; median GCS = 6) entered a standardized rehabilitation program. Functional Independent Measure scores were measured yearly, until year 7. We classified the functional trajectories in clusters, through a latent class model. We performed single-subject prediction of trajectory membership in cases unseen during model fitting. Four trajectory types were identified (post.prob. > 0.95): high-start fast (N = 92), low-start fast (N = 168), slow (N = 130) and non-responders (N = 210). Fast responders were older (chigh = 1.8; clow = 1.1) than non-responders and suffered shorter coma (chigh = −14.7; clow = −4.3). High-start fast-responders had shorter length of stay (c = −1.6), and slow responders had lower incidence of epilepsy (c = −1.4), than non-responders (p < 0.001). Single-subject trajectory could be predicted with high accuracy at first discharge (accuracy = 0.80). In conclusion, we stratified patients based on the evolution of their response to a specific treatment program. Data at first discharge predicted the response over 7 years. This method enables early detection of the slow responders, who show poor post-acute functional gains, but achieve recovery comparable to fast responders by year 7. Further external validation in other rehabilitation programs is warranted

Combining Multimodal Information for Metal Artefact Reduction:An Unsupervised Deep Learning Framework

Metal artefact reduction (MAR) techniques aim at removing metal-induced noise from clinical images. In Computed Tomography (CT), supervised deep learning approaches have been shown effective but limited in generalisability, as they mostly rely on synthetic data. In Magnetic Resonance Imaging (MRI) instead, no method has yet been introduced to correct the susceptibility artefact, still present even in MAR-specific acquisitions. In this work, we hypothesise that a multimodal approach to MAR would improve both CT and MRI. Given their different artefact appearance, their complementary information can compensate for the corrupted signal in either modality. We thus propose an unsupervised deep learning method for multimodal MAR. We introduce the use of Locally Normalised Cross Correlation as a loss term to encourage the fusion of multimodal information. Experiments show that our approach favours a smoother correction in the CT, while promoting signal recovery in the MRI.Comment: Accepted at IEEE International Symposium on Biomedical Imaging (ISBI) 202

Automated postoperative muscle assessment of hip arthroplasty patients using multimodal imaging joint segmentation

BACKGROUND AND OBJECTIVE: In patients treated with hip arthroplasty, the muscular condition and presence of inflammatory reactions are assessed using magnetic resonance imaging (MRI). As MRI lacks contrast for bony structures, computed tomography (CT) is preferred for clinical evaluation of bone tissue and orthopaedic surgical planning. Combining the complementary information of MRI and CT could improve current clinical practice for diagnosis, monitoring and treatment planning. In particular, the different contrast of these modalities could help better quantify the presence of fatty infiltration to characterise muscular condition and assess implant failure. In this work, we combine CT and MRI for joint bone and muscle segmentation and we propose a novel Intramuscular Fat Fraction estimation method for the quantification of muscle atrophy.METHODS: Our multimodal framework is able to segment healthy and pathological musculoskeletal structures as well as implants, and develops into three steps. First, input images are pre-processed to improve the low quality of clinically acquired images and to reduce the noise associated with metal artefact. Subsequently, CT and MRI are non-linearly aligned using a novel approach which imposes rigidity constraints on bony structures to ensure realistic deformation. Finally, taking advantage of a multimodal atlas we created for this task, a multi-atlas based segmentation delineates pelvic bones, abductor muscles and implants on both modalities jointly. From the obtained segmentation, a multimodal estimation of the Intramuscular Fat Fraction can be automatically derived.RESULTS: Evaluation of the segmentation in a leave-one-out cross-validation study on 22 hip sides resulted in an average Dice score of 0.90 for skeletal and 0.84 for muscular structures. Our multimodal Intramuscular Fat Fraction was benchmarked on 27 different cases against a standard radiological score, showing stronger association than a single modality approach in a one-way ANOVA F-test analysis.CONCLUSIONS: The proposed framework represents a promising tool to support image analysis in hip arthroplasty, being robust to the presence of implants and associated image artefacts. By allowing for the automated extraction of a muscle atrophy imaging biomarker, it could quantitatively inform the decision-making process about patient's management.</p

A multi-channel uncertainty-aware multi-resolution network for MR to CT synthesis

Synthesising computed tomography (CT) images from magnetic resonance images (MRI) plays an important role in the field of medical image analysis, both for quantification and diagnostic purposes. Convolutional neural networks (CNNs) have achieved state-of-the-art results in image-to-image translation for brain applications. However, synthesising whole-body images remains largely uncharted territory, involving many challenges, including large image size and limited field of view, complex spatial context, and anatomical differences between images acquired at different times. We propose the use of an uncertainty-aware multi-channel multi-resolution 3D cascade network specifically aiming for whole-body MR to CT synthesis. The Mean Absolute Error on the synthetic CT generated with the MultiResunc network (73.90 HU) is compared to multiple baseline CNNs like 3D U-Net (92.89 HU), HighRes3DNet (89.05 HU) and deep boosted regression (77.58 HU) and shows superior synthesis performance. We ultimately exploit the extrapolation properties of the MultiRes networks on sub-regions of the body